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Air  Force  Research  Laboratory  (AFRL)  researchers  at  the  Aerospace  Vehicles 
Technology  Assessment  and  Simulation  (AVTAS)  Laboratory  are  developing  a  realistic 
urban  simulation  environment.  The  near  term  objective  is  to  provide  an  appropriate 
environment  to  study  the  performance  of  cooperative  control  algorithms  for  Unmanned  Air 
Vehicles  (UAVs)  in  and  around  the  urban  landscape.  Additionally,  operator-in-the-loop 
interfaces  that  interact  with  the  cooperative  control  algorithms  will  be  implemented 
providing  the  capability  to  explore  the  various  facets  of  UAV  control.  The  simulation 
environment  will  include  multiple  urban  databases  for  visualization,  UAV  aerodynamics  and 
control  models,  camera  models,  articulated  human  models,  and  ground  vehicle  models  to 
serve  as  clutter  and  potential  targets.  Wind  and  turbulence  models  will  be  also  be  integrated 
to  elicit  realistic  UAV  behavior.  The  levels  of  fidelity  can  be  varied  depending  on  available 
resources  and  design  of  the  experimental  study. 


Nomenclature 


ATR 

AVTAS 

DTED 

GCS 

GPS 

INS 

OVI 

UAV 

USE 


Automatic  Target  Recognition 

Aerospace  Vehicles  Technology  Assessment  and  Simulation 

Digital  Terrain  Elevation  Data 

Ground  Control  Station 

Global  Positioning  System 

Inertial  Navigation  System 

Operator  Vehicle  Interface 

Unmanned  Air  Vehicle 

Urban  Simulation  Environment 


I.  Introduction 

The  objective  of  the  Urban  Simulation  Environment  (USE)  is  to  provide  a  flexible  tool  for  researchers  to  execute 
simulations  involving  air  vehicles  in  a  representative  urban  setting.  A  representative  urban  setting  would  be  an 
environment  that  has  objects  typically  found  in  urban  situations.  This  would  include  such  items  as  buildings, 
structures  such  as  telephone  or  light  posts,  people,  and  vehicles.  Other  environments  may  contain  these  items,  but 
an  urban  environment  is  unique  in  that  the  density  of  these  objects  is  much  greater  than  a  rural  or  suburban 
environment.  The  environment  could  incorporate  other  models,  such  as  turbulence  and  urban  wind  fields,  and 
Global  Positioning  System  (GPS)  communications. 

The  simulation  environment  would  not  restrict  the  type  of  air  vehicle  that  could  potentially  be  used.  Currently, 
unmanned  air  vehicles  (UAVs)  are  being  modeled  but  it  would  be  possible  to  have  piloted  vehicles  incorporated  into 
the  simulation  if  this  was  desired.  An  important  attribute  of  the  USE  is  that  it  has  been  constructed  so  that  it  will  be 
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relatively  easy  to  make  modifications.  The  interfaces  between  different  components  are  well  defined  and  based  on 
open  standards,  so  any  new  models  can  be  incorporated  by  making  them  compliant  with  the  interface. 

There  are  several  types  of  simulations  with  differing 
fidelities  and  simulation  products.  Figure  1  illustrates 
some  of  the  different  types  of  simulations.  Fundamental 
science  level  simulations  are  high  fidelity  and  produce 
Measures  of  Performance  (MOP).  An  example  of  this 
type  of  simulation  would  be  to  simulate  a  radar  system 
and  its  capability  to  detect  aircraft.  The  simulation 
would  need  to  take  into  account  the  different 
wavelengths  that  the  radar  is  emitting  and  the  reflections 
off  of  objects.  A  possible  MOP  would  be  the  range  at 
which  a  particular  type  of  aircraft  would  be  detected. 

The  second  level  of  the  simulation  hierarchy  focuses 
on  mission-level  simulations.  At  this  level,  measures  of 
Effectiveness  (MOE)  are  used  to  evaluate  how  well  a 
mission  is  accomplished.  An  example  of  a  mission 
simulation  would  be  the  collection  of  visual  data  from 
some  area  in  search  of  a  set  of  targets.  In  this  case  a 
MOE  of  this  mission  might  be  the  number  of  targets 
identified.  The  highest  level  of  simulation  is  the 
campaign  level.  At  this  level  of  simulation,  the  product  is  to  determine  the  outcome  of  a  battle  or  possibly  an  entire 
conflict.  The  USE  simulation  environment  is  not  designed  to  address  this  level  of  simulation. 

The  USE  is  an  environment  designed  for  studies  that  examine  the  effectiveness  of  UAVs  performing  missions  in 
an  urban  environment.  As  such,  the  USE  fits  the  middle  section  of  the  simulation  hierarchy.  Simulation 
components  feature  an  effects-level  fidelity,  allowing  fast  execution  while  providing  enough  detail  to  effectively 
represent  the  environment.  It  is  envisioned  that  the  products  of  the  USE  will  be  measures  of  effectiveness  of  various 
concepts  of  employment  for  teams  of  UAVs. 

II.  Motivation  for  Development  of  USE 

Urban  environments  present  a  set  of  unique  characteristics.  Primarily,  urban  environments  are  typically 
cluttered  with  various  types  of  structures  that  tend  to  obscure  aerial  views.  They  are  often  also  populated  with 
people  and  ground  vehicles.  This  presents  a  higher  probability  of  collateral  damage.  Furthermore,  there  is  typically 
critical  infrastructure  in  the  environment,  such  as  bridges,  power  stations,  or  communication  nodes  that  may  play  an 
important  part  in  a  simulation. 

One  of  the  characteristics  of  an  urban  environment  is  that  the  aerial  views  of  the  urban  terrain  are  obscured  from 
many  different  angles  with  respect  to  the  air  vehicle’s  position.  Streets  are  typically  lined  with  many  tall  buildings, 
which  create  a  canyon  effect.  An  air  vehicle’s  position  with  respect  to  the  canyon  will  dramatically  affect  what  can 
be  viewed.  For  example,  if  a  person  or  vehicle  model  is  a  subject  of  interest  and  is  located  on  a  street  in  the  canyon, 
then  a  UAV  may  or  may  not  be  able  to  have  a  line  of  sight.  If  the  air  vehicle  has  a  view  down  the  street  or  is  high 
enough,  then  it  may  be  possible  to  view  the  subject  of  interest.  However,  if  the  air  vehicle  is  trying  to  view  across 
the  canyon,  the  canyon  could  be  too  deep  to  provide  a  view  of  the  subject.  The  intent  of  the  USE  environment  is  to 
provide  researchers  a  simulation  environment  where  most  of  the  effects  of  the  urban  environment  will  be  modeled. 
This  includes  the  obscuration  of  visual  sensor  views  due  to  urban  structures  blocking  particular  aerial  viewpoints. 

An  example  research  topic  could  look  at  how  to  overcome  the  canyons  in  an  urban  environment  by  using  a 
cooperative  control  strategy  of  controlling  multiple  UAVs.  If  one  UAV  is  unable  to  view  the  subject  of  interest, 
then  the  strategy  could  try  to  assign  the  task  to  another  UAV.  Research  will  also  need  to  be  done  to  evaluate  the 
optimum  amount  of  information  that  can  be  presented  to  one  or  more  operators  viewing  UAV  sensor  video  streams. 
If  you  try  to  present  too  much  information  to  an  operator,  then  they  may  be  unable  to  effectively  monitor  all  of  the 
incoming  data.  The  current  limitations  are  unknown,  but  can  be  discovered  through  exercises  in  the  USE. 

Another  characteristic  of  the  urban  environment  is  that  there  are  many  people,  ground  vehicles  and  critical 
infrastructure  within  a  very  condensed  area.  The  people  can  be  of  various  classes.  They  can  be  friendly  forces, 
enemy  forces,  or  civilian.  It  is  possible  to  classify  the  people  into  other  types  of  categories  depending  on  the  intent 
of  the  simulation.  The  categories  can  be  further  refined  as  necessary  but  the  USE  environment  primarily  has  people 
models  in  order  to  provide  realism.  At  this  development  point  in  the  USE  simulation  there  is  a  very  limited 


of  Outcome 


Figure  1.  Different  simulation  levels. 

A  simulation ’s  fidelity  determines  the  product  of  tests. 
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application  of  people  models.  The  people  models  in  the  urban  database  serve  as  clutter  objects  that  can  potentially 
distract  an  operator  monitoring  streaming  video  originating  from  a  simulated  UAV  sensor  camera  model. 

At  some  future  point,  the  role  of  people  models  may  become  more  significant.  There  are  many  potential 
examples  of  simulation  people  models  taking  on  a  more  elaborate  and  complex  roles.  The  simulated  people  models 
may  detect  a  UAV  flying  within  detection  range  and  take  some  action  based  on  this  detection.  Another  more 
complex  role  for  simulated  people  models  would  be  performing  activities  that  would  allow  an  observer  to  determine 
the  people  models  intent  and  allow  the  UAV  sensor  operator  the  capability  to  categorize  the  people  models  as 
civilian,  enemy,  or  friendly  forces.  An  example  would  be  a  group  of  people  models  firing  weapons  at  a  known 
friendly  force.  There  are  also  more  subtle  behaviors  that  an  operator  may  detect  on  a  video  stream  that  may  help 
determine  the  intent  of  the  people  models. 

Because  the  smaller  UAVs  might  potentially  fly  lower  to  the  ground,  the  wind  field  due  to  the  urban  structure 
becomes  a  factor.  The  modified  flow  around  the  buildings  could  affect  the  flight  paths  and  controllability  of  any 
UAVs  that  fly  in  the  proximity.  In  an  urban  environment  smaller  UAVs  could  experience  a  sudden  change  in  the 
winds  that  affects  the  UAV’s  ability  to  follow  its  flight  path.  This  deficit  of  control  has  two  distinct  effects.  First, 
there  is  the  possibility  of  collision  with  the  nearby  urban  structure.  Second,  high  levels  of  turbulence  makes  it 
difficult  to  keep  onboard  cameras  focused  on  a  particular  object.  How  much  of  an  impact  this  will  be  and  what  steps 
must  be  taken  to  work  around  the  unique  issues  of  urban  winds  have  yet  to  be  determined.  The  USE  is  being 
designed  to  help  determine  the  impact  of  these  effects. 

To  adequately  portray  all  of  the  characteristics  of  an  urban  environment,  the  simulation  environment  features  a 
visual  database  for  the  urban  environment  containing  ground  features,  urban  structure,  people  models,  and  ground 
vehicle  models.  Virtual  air  vehicles  can  fly  through  the  simulation  database  and  obtain  visualizations  representative 
of  what  would  be  seen  in  real  life.  In  order  to  assure  that  the  video  stream  from  the  UAV  is  a  realistic 
representation,  it  is  necessary  to  modify  the  simulation  video  with  representative  camera  effects  to  achieve  this 
realism.  The  specific  camera  effects  that  are  applied  will  be  discussed  in  a  later  section. 

III.  USE  Architecture 


Figure!.  Block  diagram  of  USE. 

The  architecture  of  the  USE  can  be  divided  into  4  separate  areas;  the  ground  units  and  terrain,  the  flight 
dynamics  models  for  the  UAVs,  scene  generation,  and  the  operator  station.  Each  of  these  will  be  discussed  in  this 
section  along  with  a  discussion  of  the  future  capabilities  in  development. 
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A.  Overview 

The  USE  currently  consists  of  seven,  dual-processor  computers  with  both  a  gigabit  Ethernet  and  reflective 
memory  network.  Figure  2  shows  a  block  diagram  of  the  current  configuration.  One  computer  is  dedicated  to  the 
Ground  Control  Station.  During  the  simulation  execution  this  is  where  the  UAV  operator  would  have  access  to 
control  the  UAVs.  Another  computer  controls  the  movement  of  ground  vehicles  and  people  models.  A  third 
computer  is  dedicated  to  running  the  flight  models  of  the  UAVs  and  is  also  the  communication  hub  for  the 
simulation.  The  UAV  computer  receives  commands  and  returns  status  reports  to  the  ground  control  station.  It  also 
receives  the  location  of  all  of  the  entities  in  the  environment,  which  is  passed  to  the  scene  generation  computers. 
There  are  four  scene  generation  computers  which  simulate  the  video  feeds  coming  from  the  UAVs.  This  video  feed 
is  then  passed  back  to  the  ground  control  station. 

B.  Ground  Units  and  Terrain 

The  USE  was  developed  as  a  relatively  generic  simulation  environment  to  host  many  simulations.  The  key 
difference  between  this  environment  and  other  simulations  is  the  incorporation  of  detailed  surface  objects.  Other 
simulations  conducted  at  the  AVTAS  facility  are  conducted  at  higher  altitudes  and  do  not  focus  on  the  terrain  as 
much.  The  terrain  is  there  to  provide  realism;  the  players  need  it  to  get  a  visual  cue  as  to  their  velocity  and 
orientation.  Since  most  of  the  USE’s  current  work  is  conducted  at  a  much  lower  altitude,  the  details  of  the  terrain 
play  a  much  larger  role.  The  databases  developed  for  USE  generally  consist  of  small  urban,  suburban,  and  rural 
areas  of  less  than  100  square  miles.  The  detail  includes  buildings,  vehicles,  vegetation,  roads,  bodies  of  water, 
detailed  terrain,  and  in  some  cases  models  of  individual  people. 

The  focus  of  the  environment  is  to  provide  an  appropriate  level  of  stimulus  to  operators  that  are  viewing  the 
environment  via  a  simulated  air  vehicle.  Often  the  operator  is  tasked  with  finding  a  potential  enemy  in  an  area  of 
civilian  clutter.  The  challenge  is  to  provide  the  operator  with  a  level  of  detail  that  would  allow  for  an  operator  to 
distinguish  between  bystanders  and  foes.  Some  of  these  details  include  clothes  and  facial  expressions.  Their  actions 
would  also  need  to  be  taken  into  account.  The  intent  of  individuals  can  be  detected  much  more  readily  through 
dynamic  activities  rather  than  a  group  of  static  individuals. 

To  animate  the  ground  vehicles  and  people,  the  USE  uses  the  Flexible  Analysis  Modeling  and  Exercise  System 
(FLAMES)  \  FLAMES  provides  many  usehil  tools  as  a  simulation  architecture  such  as  built  in  communications 
between  models  and  the  ability  to  load  in  terrain  databases.  The  terrain  database  is  loaded  through  shapefiles"^. 
These  shapefiles  contain  both  the  terrain  elevation  and  correlated  features.  There  are  named  features  such  as  roads, 
buildings  and  rivers.  This  allows  the  ground  vehicles  to  know  the  location  of  the  buildings  and  roads.  Each  ground 
vehicle  can  then  be  given  a  route  to  follow  which  can  consist  of  coordinates  or  even  street  names.  The  ability  for  the 
vehicles  to  roam  randomly  through  the  urban  database  is  in  development,  but  will  enhance  the  realism  of  the  virtual 
urban  environment  by  enhancing  the  background  clutter. 


Figure  3.  Screenshots  of  one  person  and  a  truck.  These  are  a  sample  of  the  currently  available  models  for  the 
environment. 
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Detailed  3-D  ground  entities  such  as  human  models  and  an  urban  vehicle  set  were  needed  for  scene  generation. 
Figure  3  shows  some  of  these  models  in  the  USE.  A  base  model  set  was  purchased  for  the  vehicle  models  which 
included  ten  different  model  types  with  six  different  skin  colors.  These  models  included  sedans,  coupes  and  pickup 
trucks.  Additionally,  several  unique  models  were  created  and  added  to  the  base  target  set,  such  as  an  ambulance 
with  particular  markings.  Articulation  was  added  to  the 
human  models  to  represent  basic  human  motions  such  as 
running,  walking,  sitting  and  crawling.  Using  IEEE 
1278.1  enumerations  for  human  position  states,  the 
visual  model  can  be  put  into  any  of  these  states  in  the 
visual  scene.  The  selection  of  which  particular  state  an 
individual  is  in  at  any  given  time  is  driven  by  FLAMES 
and  is  passed  along  with  the  entities  location  to  the  scene 
generation  system. 


C.  Scene  Generation 

Scene  generation  is  managed  by  using  two  AFRL- 
developed  software  packages;  the  Virtual  Battle  Space 
Management  System  (VBMS)  and  SubrScene. 

The  Virtual  Battle-Space  Management  System 
(Figure  4)  is  a  flexible  tool  used  for  visualizing 
simulation  environments.  VBMS  acts  as  an  observer  in 
the  simulation,  receiving  data  in  order  to  record  entity 
interactions  in  the  simulation,  but  does  not  interact  or 
interfere  with  simulation  entities.  VBMS  can  be  used  for 
validation,  monitoring,  out-the-window  visualization  for 
pilot  combat  stations,  subject  debriefing,  and  analysis. 

The  C++  design  is  built  around  an  OpenGL  based  scene  manager,  which  adds  a  cost  effective  graphics  library  for 
fast  prototyping  with  optimized  performance  and  allows  VBMS  to  operate  on  the  full  line  of  workstation  and 
personal  computers.  VBMS  uses  2D  and  3D  visualization  to  view  the  synthetic  environment.  In  addition  to 
standard  capabilities  for  VBMS,  plug-ins  can  be  created  to  enhance  the  tool.  The  SiteBuilder  plug-in  was  created  to 
aid  in  the  scenario  development  of  the  simulation.  This  plug-in  allows  the  user  to  interactively  place  entities 
anywhere  in  the  environment.  Ground  routes  can  be  added  by  drawing  them  out.  Through  the  logging  capability, 
simulation  runs  can  then  be  used  to  fine  tune  entity  placement  and  routes.  All  data  can  be  exported  back  into  native 

simulation  formats  to  run  scenarios. 
As  a  post-run  tool,  recorded 
simulation  runs  are  taken  for  debrief, 
and  AVI  capture  for  final  briefings. 

The  primary  scene  generation  tool 
is  SubrScene.  In  Figure  2,  SubrScene 
is  the  scene  generation  software  that 
produces  the  imagery  as  seen  by  each 
UAV.  Specific  requirements  for 
sensor  emulation  of  various  UAVs 
were  required  in  the  USE.  Data  was 
collected  about  each  sensor  including 
line  rates,  transmission  formats, 
integration  time  of  image,  and  field  of 
view  angles.  In  addition,  information 
on  sensor  mounting,  dynamic 
positioning  and  zoom  were  recorded. 
Sample  video  from  the  sensor  was 
collected  and  characterized  based  on 
signal  to  noise  effects,  and 
environment  effects  such  as  sun 
blooming.  For  the  sensor  of  interest, 
effects  from  the  sun,  NTSC  line 


Figure  5.  Representative  Urban  Area.  Sample  image  of  3-D  visuals. 
Terrain  is  built  from  aerial  imagery  with  3-D  models  placed  on  top. 


Figure  4.  Screenshot  of  VBMS. 

Configuration  here  is  shown  with  the  SiteBuilder 
plugin,  which  is  used  to  place  vehicles  in  the 
environment. 
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reduction,  integration  time  and  sensor  blooming 
caused  by  the  sun  to  sensor  interaction  were 
needed.  For  this  simulation  an  individual  copy  of 
Subr Scene  was  used  for  each  sensor.  A  plug-in  was 
created  for  the  characterized  sensor  to  emulate  the 
effects  characterized.  Communication  from  the 
simulation  drove  values  such  as  position,  field  of 
view,  sensor  modes  and  sensor  data  loss  due  to  line 
of  sight  and  interference.  This  video  image  was 
then  transmitted  via  reflective  memory  network 
cards  to  the  sensor  operator  station  where  the 
various  configurations  allowed  operator  interaction 
with  the  sensors.  A  frame  of  video  showing  signal 
loss  effects  is  shown  in  Figure  6. 

The  visual  database  was  created  by  translating 
elevation  data  obtained  through  DTED  level  1  data. 
The  elevation  data  was  combined  with  0.5  meter 
aerial  imagery  to  produce  the  virtual  scene.  Culture, 
such  as  buildings,  roads,  and  bodies  of  water,  were 
extracted  manually  from  the  imagery  and  then 
added  back  in  as  3-D  models.  The  database  is  correlated  with  FLAMES  to  ensure  that  entities  controlled  by 
FLAMES  would  have  identical  locations  with  the  visual  scene.  An  example  of  a  completed  database  is  shown  in 
Figure  5. 

D.  Ground  Control  Station 

The  ground  control  station  is  the  human  operator’s  interface  to  the  simulation.  The  GCS  is  capable  of  sending 
commands  and  receiving  data  from  the  UAVs,  such  as  waypoint  following  commands,  loitering,  or  sensor  requests. 
The  GCS  receives  data  from  UAVs  regarding  location,  fuel  status,  and  any  available  sensor  data,  such  as  a  video 
stream  or  still  images. 

Although  many  ground  control  stations  exist,  the  USE  employs  an  engineering-level  station  that  was  written  by 
engineers  at  the  AVTAS  facility.  This  system,  dubbed  the  Engineering  Level  Vehicle  Interface  System  (ELVIS),  is 
used  as  a  development  and  testing  tool.  It  is  not  designed  to  be  used  in  the  field,  but  it  does  provide  utility  in 
exercising  the  system.  Since  it  was  developed  in  house,  AVTAS  engineers  are  able  to  add  new  features  quickly. 
Communication  with  air  vehicles  is  via  an  Ethernet  packet  scheme  that  is  based  on  NATO  STANAG  4586.  An 
example  of  ELVIS  is  shown  in  Figure  7. 

Video  links  between  the  GCS  and  the  air 
vehicles  are  also  modeled  in  the  USE.  When 
implementing  the  video  stream,  two  different 
approaches  were  considered.  One  approach 
would  use  a  separate  computer  with  video 
capture  boards  to  capture  the  data  and  send  it 
through  Ethernet  to  the  operator  station.  The 
other  approach  used  reflective  memory  cards  to 
quickly  send  the  video  to  the  operator  station. 

A  reflective  memory  card  has  on-board 
memory  and  all  of  the  boards  on  their  network 
have  exactly  the  same  data.  This  means  that  if 
a  value  is  written  to  a  memory  location  on  one 
board  that  same  value  will  be  on  every  other 
board.  This  allows  video  to  be  copied  out  of 
the  frame  buffer  on  the  scene  generation 
computers  and  sent  directly  to  the  operator 
station.  AVTAS  engineers  found  the  later 
implementation  to  be  the  better  approach. 

Using  the  video  capture  boards  has  some  benefits,  but  many  drawbacks.  The  biggest  benefit  would  be  to  lessen 
the  processing  burden  on  the  scene  generation  computer  of  copying  each  frame.  However,  since  all  of  the  video 
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Figure  7.  ELVIS  Screenshot 
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capture  boards  were  in  the  same  computer,  it  was  limited  by  the  bandwidth  on  the  PCI  bus  which  connects 
peripherals  to  the  CPU.  All  of  the  video  streams  that  are  being  captured  have  to  share  that  bus,  which  means  that  the 
video  has  to  be  compressed  more  to  allow  them  all  to  fit.  This  could  add  unintended  artifacts  to  the  video.  Also, 
when  sending  video  over  the  network,  the  receiving  end  has  to  buffer  the  video  before  playback.  This  caused  a  one- 
second  delay  in  the  video,  making  it  impossible  to  correctly  simulate  the  video  feed.  There  may  be  cases  where 
delays  would  be  desirable,  perhaps  to  model  transmission  delays,  but  if  the  calculated  delay  is  less  than  a  second, 
then  it  would  be  impossible.  Because  of  these  limitations,  the  USE  team  looked  to  reflective  memory  as  an 
alternative  method  of  video  transmission. 

As  mentioned  previously,  the  reflective  memory  boards  provide  common  memory  storage  between  multiple 
computers.  This  enables  the  ability  to  copy  the  scene  from  the  scene  generation  computer  directly  to  the  operator 
station.  This  reduces  the  delay  to  approximately  a  microsecond.  Additionally,  the  higher  bandwidth  of  reflective 
memory  allows  for  the  modeling  of  higher  resolution  cameras.  This  additional  bandwidth  allows  for  video 
resolution  studies,  whereby  various  resolutions  can  be  presented  to  operators  to  help  determine  an  adequate  video 
resolution  for  the  prescribed  task. 


E.  Flight  Dynamics  Models 

Currently,  the  flight  dynamics  model  for  the  UAVs  is  based  on  non-linear,  6-degrees  of  freedom  equations  of 
motion^.  It  is  configurable  and  cross-platform  compatible.  There  are  two  tables  that  are  loaded  into  the  model  to 
determine  the  aerodynamic,  physical,  and  control  system  parameters  for  the  model.  Since  the  model  is  not  based  on 
any  particular  system,  new  air  vehicles  can  be  modeled  by  inserting  different  data  sets.  The  USE  environment  was 
constructed  in  a  manner  that  allows  for  new  flight  models  to  be  rapidly  integrated.  This  allows  a  single  executable 
to  represent  different  UAVs,  which  provides  the  benefit  that  all  of  the  other  processes  that  allow  the  UAV  to  interact 
with  the  outside  world  can  stay  the  same.  There  are  multiple  built-in  flight  control  systems  available  that  range  from 
low-level  control  surface  commands  to  full  autopilot  control  that  commands  vehicles  through  waypoints.  For  all 
planned  USE  studies,  the  UAVs  are  assigned  waypoints,  so  the  current  flight  model  capabilities  are  sufficient. 

The  processing  of  the  flight  dynamics  models  is  controlled  by  an  in-house  developed  real-time  executive  called 
REX^.  REX  has  the  capability  to  run  processes  as  a  synchronous  or  asynchronous  task.  A  synchronous  task,  such 
as  the  flight  dynamics  model,  is  run  with  a  set  time  step.  Every  one-hundredth  of  a  second,  REX  tells  the  flight- 
dynamics  models  to  run  and  calculate  a  new  position  based  on  the  forces  applied  to  it.  Some  of  the  asynchronous 
tasks  include  the  interfaces  for  the  UAV  to  communicate  with  the  GCS,  the  scene  generation  system  and  the 
FLAMES  environment.  These  interfaces  need  to  wait  on  the  other  side  to  send  or  receive  data,  so  they  are  not 
synchronized  with  the  processing  of  the  flight  dynamics  models. 

Weather  effects  are  often  ignored  when  dealing  with  larger,  faster  air  vehicles  that  operate  at  thousands  of  feet  in 
altitude.  However,  in  urban  simulations,  these  effects  have  a  significant  impact  on  operations.  When  navigating 
urban  terrain,  small  air  vehicles  can  be  significantly  affected  by  even  low  wind  speeds.  This  is  especially  true  when 
an  operator  is  attempting  to  view  a  specific  point  on  the  ground  or  an  air  vehicle  is  being  maneuvered  around  a 
structure.  The  USE  includes  wind  modeling  as  part  of  its  architecture.  Models  of  steady-state  winds  and  turbulence 
are  inputs  to  the  flight  dynamics  models.  Currently  this  is  done  via  a  wind-field  file  lookup  method.  The  current 
wind  models  do  not  take  into  account  the  interaction  between  wind  fields  and  structure,  such  as  those  shown  in 
Figure  8.  The  USE  team  is  considering  CFD-based  models  that  incorporate  the  terrain  and  ground  structures.  As 
with  the  flight  control  system,  the  turbulence  can  be 
replaced  with  a  higher  fidelity  model  when  needed. 

F.  Future  Capabilities 

Some  of  the  capabilities  that  the  USE  team  is 
currently  developing  include  a  global  positioning 
system/inertial  navigation  system  model  and  a  better 
communication  model. 

In  the  current  operation  of  the  simulation,  the 
UAV  reports  back  to  the  operator  station  its  exact 
location,  which  does  not  accurately  portray  the 
capabilities  of  small  UAVs.  As  part  of  the  flight 
dynamic  model,  it  uses  the  equations  of  motion  to 
calculate  a  change  in  position  based  on  the  applied 

forces.  Real-world  air  vehicles  use  GPS  or  Inertial  figure  8.  MOUT  Site  Wind  Field.  Computer  simulation 
Navigation  System  (INS)  devices  to  calculate  their  buildings  place  on  winds.  (Source  AFRL/VACA). 
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position,  which  experience  errors,  drop-outs,  and  jamming. 

Communication  models  are  also  under  development.  This  would  come  into  play  between  the  operator  station 
and  the  UAV.  Currently,  there  is  an  unlimited  amount  of  bandwidth  and  no  loss  of  signal.  But  realistically,  this 
would  not  be  the  case.  By  modeling  limited  bandwidth  and  signal  loss,  the  amount  and  quality  of  data  that  can  be 
passed  between  the  GCS  and  the  UAVs  will  more  accurately  depict  real-world  scenarios.  This  would  affect  the 
amount  of  video  data  passed  as  well  as  introducing  more  realistic  confusion  and  uncertainty. 

IV.  Multi  Video  Stream  Study 

The  USE  simulation  capability  is  being  used  by  Air  Forces  Research  Laboratory  Human  Effectiveness 
Directorate  to  evaluate  the  effectiveness  of  operators  viewing  multiple  simultaneous  video  streams.  Initially,  testing 
will  involve  the  monitoring  of  multiple  video  streams  with  the  additional  aid  of  an  automatic  target  recognition 
(ATR)  that  places  an  indicator  around  objects  of  interest.  The  USE  is  very  well  suited  to  this  type  of  task,  since  a 
controlled  experiment  can  be  constructed  using  virtual  worlds.  Experiments  can  be  made  to  be  repeatable  and 
deterministic.  Similar  real-world  trials  would  have  variable  lighting  and  atmospheric  conditions  that  would  make 
such  experiments  more  difficult  to  manage.  Having  control  over  the  virtual  world  also  enables  engineers  to  design 
urban  environments  of  varying  levels  of  complexity. 

The  study  will  focus  on  the  ability  of  the  operator  to  identify  targets  amid  a  cluttered  environment.  Up  to  four 
simultaneous  streams  are  presented  to  the  operator.  It  is  envisioned  that  data  collected  by  this  study  will  aid  in  the 
development  and  implementation  of  ATR  technologies,  and  will  serve  as  a  baseline  performance  study. 

The  first  spiral  will  examine  the  issues  with  multiple  video  streams  and  an  ATR  to  highlight  potential  targets. 
The  participant  will  have  to  monitor  each  stream  in  real-time  and  will  only  have  the  time  that  the  target  is  in  the  field 
of  view  to  identify  the  target.  Participants  will  be  given  a  description  of  targets,  but  they  will  not  be  given  an 
example  picture  of  the  target.  In  these  sessions,  the  UAV  will  start  at  the  same  location  and  fly  a  preset  path. 
Participants  will  note  the  location  of  targets  in  the  video  windows  via  a  mouse  click.  A  set  of  scenarios  will  vary 
target  placement,  number  of  video  streams  and  the  level  assistance  from  an  automatic  target  recognition  algorithm. 
Randomizing  the  location  of  the  targets  will  prevent  memorization  of  target  location  by  participants.  The  targets  are 
also  placed  so  that  the  operator  may  or  may  not  have  multiple  targets  in  the  streams  at  the  same  time. 

The  number  of  video  streams  presented  to  the  participant  will  vary  between  1,  2,  and  4  simultaneous  streams  in 
order  to  vary  the  amount  of  workload  by  the  operator.  Without  any  aid,  an  increase  in  the  number  of  streams  should 
result  in  a  decrease  in  the  performance  of  the  operator. 

An  automatic  target  recognition  algorithm  is  set  up  to  identify  a  given  percentage  of  targets  in  the  scenario.  Two 
rates  have  been  selected  for  this  study:  70%  and  90%.  The  ATR  will  also  have  a  one  percent  false  alarm  rate  that 
will  identify  one  percent  of  all  vehicles,  not  just  targets,  as  a  potential  target.  This  will  add  additional  workload, 
since  the  operator  can  not  rely  on  the  ATR  identifying  targets  automatically. 

Targets  will  be  pickup  trucks  similar  to  that  shown  in  Figure  3.  The  target  vehicles  will  be  interspersed  among 
non-target  vehicles.  It  was  determined  that  the  trucks  present  a  distinguishable  difference  from  other  vehicles  while 
still  requiring  a  significant  level  of  workload  for  identification. 

V.  Conclusion 

Urban  environments  are  an  area  where  effective  real-world  tests  are  difficult  to  conduct.  Concern  for  public 
safety  and  airspace  management  problems  preclude  testing  in  cities.  A  simulation  environment  such  as  the  USE 
eliminates  any  concerns  about  safety  or  of  lost  aircraft,  while  providing  a  cost  effective  testing  solution.  Given  the 
special  characteristics  of  urban  environments,  the  USE  will  likely  be  used  to  conduct  future  simulations  involving 
different  types  of  technologies  such  as  sensors,  air  vehicles  types,  cooperative  control  strategies,  and  operator 
interfaces.  Being  a  simulation  environment,  it  is  relatively  easy  to  modify  parameters  of  the  models  and  execute 
parametric  studies  to  determine  the  effectiveness  of  different  technologies  or  mixes  of  technologies. 
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